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ABSTRACT 


The existing literature suggests there are no significant outcome differences 
between online and traditional degree programs in the civilian sector. Few studies have 
looked for such differences within military schools and colleges, specifically. Given the 
growing popularity of online and distance education degree programs, we study the 
impact of this particular mode of instructional delivery on the academic and subsequent 
job performance of military officer students enrolled at the Naval Postgraduate School 
(NPS). Using propensity score matching, we estimate the effects that being a distance 
learning (DL) student has on four performance outcomes: grade point average, 
graduation, promotion, and separation. We further subdivide the sample into various 
subgroups based on military service branch, warfare community, academic preparation, 
and school within NPS to determine the heterogeneous effects of DL within each 
subsample. The DL students studied performed significantly worse than equivalent 
resident students on every measurement. We found NPS students enrolled in DL degree 
programs obtain GPAs approximately half a letter grade lower, are less likely to graduate, 
are less likely to promote, and are more likely to separate from military service than their 
NPS resident student counterparts. Given these results, it is imperative to conduct 
additional research to ascertain what makes distance learning inferior to residency at the 
Naval Postgraduate School. 


V 



THIS PAGE INTENTIONALLY LEET BLANK 


VI 



TABLE OF CONTENTS 


I. INTRODUCTION.I 

A. SCOPE OF THIS THESIS.2 

B. RESEARCH QUESTIONS.2 

C. ORGANIZATION OF THIS THESIS.3 

II. LITERATURE REVIEW.5 

A. META-ANALYSES.5 

B. OBSERVATIONAL STUDIES.7 

C. RANDOMIZED STUDIES.10 

III. DATA/METHODOLOGY.13 

A. NPS DATA.13 

1. Sample.13 

2. Independent Variables.13 

a. Treatment Indicator . 13 

b. NPS Institutional Controls . 14 

c. Academic Preparation . 14 

d. Service and Community . 15 

3. Dependent Variables.16 

B. DMDC DATA.16 

1. Sample.16 

2. Independent Variables.16 

3. Dependent Variables.16 

C. IPEDS DATA.17 

1. Sample.17 

2. Variables.17 

D. DATA SUMMARY.18 

E. METHODOLOGY.26 

1. Stage One.26 

2. Stage Two.28 

IV. RESULTS.31 

A. STAGE ONE.31 

B. STAGE TWO.32 

C. HETEROGENEITY.35 

1. Service.36 

2. Community.36 

vii 





































3. Rank.37 

4. APC.38 

5. Sector.38 

6. School.39 

V. SUMMARY AND CONCLUSIONS.41 

A. SUMMARY.41 

B. RECOMMENDATIONS.41 

APPENDIX A. PROPENSITY SCORE MATCHING-LOGIT RESULTS.43 

APPENDIX B. OLS REGRESSION RESULTS FOR THE OUTCOME TQPR.45 

APPENDIX C. LPM REGRESSION RESULTS FOR THE OUTCOME 

GRADUATE.47 

APPENDIX D. LPM REGRESSION RESULTS FOR THE OUTCOME 

PROMOTED.49 

APPENDIX E. LPM REGRESSION RESULTS FOR THE OUTCOME 

SEPARATED.51 

APPENDIX F. OLS REGRESSION RESULTS FOR THE OUTCOME 

TQPR—HETEROGENEITY.53 

APPENDIX G. LPM REGRESSION RESULTS FOR THE OUTCOME 

GRADUATE—HETEROGENEITY.55 

APPENDIX H. LPM REGRESSION RESULTS FOR THE OUTCOME 

PROMOTED—HETEROGENEITY.57 

APPENDIX I. LPM REGRESSION RESULTS FOR THE OUTCOME 

SEPARATED—HETEROGENEITY.59 

LIST OF REFERENCES.61 


INITIAL DISTRIBUTION LIST 


,63 





















LIST OF FIGURES 


Figure 1. Military service branch comparison for DL and residency.20 

Figure 2. Warfare community comparison for DL and residency.21 

Figure 3. Military rank comparison for DL and residency.21 

Figure 4. NFS school comparison for DL and residency.22 

Figure 5. APC 1 undergraduate GPA comparison for DL and residency.22 

Figure 6. APC 2 undergraduate mathematics background comparison for DL 

and residency.23 

Figure 7. APC 3 undergraduate science and technical background comparison 

for DL and residency.23 

Figure 8. Undergraduate institution sector comparison for DL and residency.24 

Figure 9. Cender comparison for DL and residency.24 

Figure 10. Marital status comparison for DL and residency.25 

Figure 11. Race comparison for DL and residency.25 

Figure 12. Example propensity score matching overlay.27 

Figure 13. Propensity score matching overlay.32 


IX 
















THIS PAGE INTENTIONALLY LEET BLANK 


X 



LIST OF TABLES 


Table 1. Service and community breakdown.15 

Table 2. IPEDS sector breakdown.17 

Tables. Sample summary.19 

Table 4. Independent variables.27 

Table 5. Average treatment effect summary.33 

Table 6. Treatment heterogeneity by service.36 

Table 7. Treatment heterogeneity by community.37 

Table 8. Treatment heterogeneity by rank.37 

Table 9. Treatment heterogeneity by APC.38 

Table 10. Treatment heterogeneity by sector.39 

Table 11. Treatment heterogeneity by school.39 














THIS PAGE INTENTIONALLY LEET BLANK 



LIST OF ACRONYMS AND ABBREVIATIONS 


APC 

DL 

DMDC 

GSBPP 

GSEAS 

GSOIS 

IPEDS 

EPM 

NCES 

NPS 

OES 

SIGS 

TUCE 


Academic Profile Code 

distance learning 

Defense Manpower Data Center 

Graduate School of Business and Public Policy 

Graduate School of Engineering and Applied Sciences 

Graduate School of Operational and Information Sciences 

Integrated Postsecondary Education Data System 

linear probability model 

National Center for Education Statistics 

Naval Postgraduate School 

Original Eeast Squares 

School of International Graduate Studies 

Test of Understanding College Economics 


xiii 



THIS PAGE INTENTIONALLY LEET BLANK 


XIV 



ACKNOWLEDGMENTS 


I would like to thank my wife, Amy. She has been nothing but patient, 
understanding, and supportive throughout this entire process. Without the calm and 
loving environment, or the occasional redirection—“You’d better get off that couch and 
get back to writing”—she provided, it’s likely I’d never have finished. 

I would also like to thank my two advisors. Dr. Marigee Bacolod and Dr. Latika 
Hartmann. I cannot even imagine how much worse this process and resulting thesis 
would have been without their guidance, help, and patience. 


XV 



THIS PAGE INTENTIONALLY LEET BLANK 


XVI 



I. 


INTRODUCTION 


The last decade has seen a growing interest in determining the impact of distance 
or online forms of education on grades and subsequent job performance. Given the lower 
cost of delivering such degrees, the motivation to move to online based programs has 
increased substantially in recent years. While these cost savings are certainly relevant in 
the civilian context, the military recognizes the potential for additional cost savings by 
not having to relocate personnel for the sole purpose of higher education. 

This popularity and cost savings have led to much academic research on the 
subject. Overall, the findings on the impact of distance or online educational class 
formats on academic performance show little to no significant difference between online 
and traditional programs. Although, existing studies do indicate that synchronous 
distance education formats are inferior to asynchronous formats and that effects tend to 
be heterogeneous with weaker students performing proportionally worse in distance 
formats. However, the current research focuses heavily on civilian institutions. This 
prompts the question: Is the impact of distance education on academic and job 
performance outcomes similar at military institutions, such as the Naval Postgraduate 
School (NPS)? 

NPS offers several online degree programs that come under the broad umbrella of 
distance learning (DL) programs. The Graduate School of Operational and Information 
Sciences (GSOIS) offers six DL master’s programs in computer science related 
curriculums. The Graduate School of Engineering and Applied Sciences (GSEAS) offers 
eleven DE master’s programs in engineering related curriculums. The Graduate School of 
Business and Public Policy (GSBPP) offers four DE master’s programs in business and 
contract management related curriculums (Naval Postgraduate School, 2016). 

In this thesis, I estimate the impact of DE on student performance and subsequent 
labor market outcomes for NPS military students. To achieve this end, I merge data from 
three sources—the Institutional Research, Reporting, and Analysis Office at NPS, the 
Defense Manpower Data Center (DMDC), and the Integrated Postsecondary Education 
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Data System (IPEDS)—to create a large sample of students with extensive information 
on their demographics, undergraduate institution, and military career progression. Using 
propensity score matching, I estimate the impact of DL on four performance outcomes: 
grade point average, graduation, promotion, and separation. I further subdivide the 
sample into various subgroups based on military service branch, warfare community, 
academic preparation, and school within NPS to determine the heterogeneous effects of 
DL within each sub sample. 

My findings indicate that DL students at NPS on average have lower grades, are 
less likely to graduate, are less likely to promote in their subsequent military career, and 
are more likely to separate from active service. Similar to the literature from the civilian 
sector, I also find heterogeneous effects of DL. However, at NPS lower-ability DL 
students do not perform worse compared to higher-ability DL students. 

A. SCOPE OF THIS THESIS 

This thesis analyzes NPS students in both DL and resident programs with 
academic year start dates between 2006 and 2013. Data for the years in question and 
provided by the Institutional Research, Reporting, and Analysis Office at NPS is merged 
with data from the Defense Manpower Data Center (DMDC) and the Integrated 
Postsecondary Education Data System (IPEDS) to supplement the range of students with 
additional demographic, undergraduate institution, and promotion information. The 
analysis addresses whether or not there is a difference between DL and resident student 
performance and career progression. 

This research is quantitative in nature. I conduct a review and evaluation of 
relevant literature with a focus on distance education and online programs and their 
perceived impact on university systems. I also use other literature to provide information 
relevant to my research topic. 

B. RESEARCH QUESTIONS 

1. What is the causal impact of distance learning on military officer students’ 
academic and job performance? 
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2. What student characteristics best predict success in distance learning vs. 
resident learning? 

3. Are there systematic differences between distance learning and resident 
military officer demographic characteristics, MOS, ability, and/or 
academic preparation? 

C. ORGANIZATION OF THIS THESIS 

This thesis consists of five chapters. Chapter I provides a brief introduction to the 
subject matter and methodology. Chapter II presents a review of relevant literature with a 
focus on the impact of distance education on performance outcomes. Chapter III 
delineates the data utilized for analysis, demonstrates difference between DL and resident 
student bodies, and provides an in depth insight into the methodology behind this thesis. 
Chapter IV presents and explains the findings of this thesis. Finally, Chapter V provides 
concluding remarks and preliminary recommendations. 
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II. LITERATURE REVIEW 


Given the recent growth in distance education degree programs within the U.S. 
military and beyond, there is an increasing need to better understand the impact of 
distance education on student learning outcomes. Hence, the large literature on the topic 
continues to grow more extensive with each passing year. The current literature falls into 
three categories: meta-analyses, observational studies, and randomized experiments. I 
summarize the main findings from these categories in order below. 

A. META-ANALYSES 

Meta-analyses are a quantitative summary of existing studies and present a 
general picture of the state of research in a particular subject area. I focus on three key 
and recent studies here that focus on how online and distance education modes of 
delivery impact outcomes. A U.S. Department of Education meta-analysis conducted by 
Means, Toyama, Murphy, Bakia, and Jones (2010) looks explicitly at online education 
versus traditional face-to-face instruction focusing on post-secondary education. Lack 
(2013) expounds upon Means et al. (2010) and provides additional insight for future 
studies in this area. Unlike these studies, Bernard et al. (2004) focus explicitly on 
distance education, separate from online education, and discuss at length the issue of 
synchronous versus asynchronous delivery methods. Additionally, there are a variety of 
outcomes used throughout the many smaller studies included in these meta-analyses, 
including individual course grades, overall grade point average, instructor evaluation 
surveys, and student evaluation surveys. However, the best and most common outcome 
chosen is individual student course grade because it allows for a degree of control 
between different classes and instructors and bypasses the more qualitative nature of 
surveys. 

Means et al. (2010) initially set out to provide information for K-12 students. 
However, the vast majority of existing studies revolve around secondary and post¬ 
secondary education, excluding any significant measures of effect for the intended 
student body (2010, p. 31). Also, they drop a large number of studies that simply 
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compare online and traditional students, as they are likely to be biased by a student’s 
selection into a particular type of course. By screening and eliminating all but 45 from a 
pool of 1,132 existing studies. Means et al. do provide several significant results relevant 
to continued education (2010, p. 14). After combining studies and weighting results 
based on respective sample sizes, they find that the online delivery method is just as 
effective as face-to-face instruction but not any better (Means et al., 2010, p. 18). Also, 
the evidence suggests that supplementing classroom instruction with online resources, 
often known as hybrid delivery, has a positive and significant impact on student 
performance as measured by grades (p. 19). Further, the types of delivery media among 
online courses demonstrate no significant effect on average learning outcomes (Means et 
al., 2010, p. 40). Means et al. (2010) also highlight the wide variation in methods, 
findings, and effect sizes across studies. The wide variation in methodologies likely 
contributes to the wide variation in findings. Also, although they eliminated studies based 
on the level of qualitative analysis and chosen outcome variables and then weighted 
based on sample size, many studies were severely biased and lacking in good quality 
control variables (Means et al., 2010, p. 13). 

Lack (2013) conducts a meta-analysis similar to that of Means et al. (2010) but 
with several key differences with regard to her focus. Lack (2013) identifies 30 studies 
that compare some combination of face-to-face, online, and/or hybrid learning. She also 
only includes studies with learning outcomes, such a course grades, as dependent 
variables and precludes studies with student authors (p. 8). Instead of attempting to 
calculate overall effects. Lack (2013) asserts that the existing literature is inadequate to 
determine whether or not online or hybrid learning modalities are more or less effective 
than traditional face-to-face modalities (p. 10). The discussion, in turn, revolves around 
the shortage of quality studies, small sample sizes, and need for random assignment 
(Lack, 2013, p. 8). In particular, this study highlights four general problems with the 
current state of research. First, those studies that achieve random assignment to distance 
education have very small sample sizes. Second, those studies with large samples often 
lack sound experimental design and display widely conflicting results. Third, existing 
observational studies generally fail to account for self-selection bias despite controlling 
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for background information. And finally, the remaining studies comprise those that 
neglect relevant controls (Lack, 2013, p. 11-12). 

Bernard et al. (2004) conduct yet another meta-analysis, although older than the 
previous two. This study focuses exclusively on distance education, which can be 
separate from online education that does not involve geographic separation. Findings 
include a small yet significant positive effect of distance education on learning outcomes 
(Bernard et al., 2004, p. 404). However, the standard errors are large, and thus the overall 
result tends to be more in accordance with Means et al. (2010) of no significant 
differences in student outcomes across delivery modes. What causes this study to stand 
out is the emphasis on the effects of synchronous and asynchronous methods of distance 
education. Synchronous distance education results in a significant negative effect on 
learning outcomes, while asynchronous distance education results in a significant positive 
effect on learning outcomes (Bernard et al., 2004, p. 404). Further, all three outcomes 
measured—achievement, attitude, and retention—yield conflicting results between 
synchronous and asynchronous formats. With respect to all three outcomes, synchronous 
formats favor the classroom environment, whereas asynchronous formats favor distance 
education (Bernard et al., 2004, p. 408). 

These large meta-analyses draw on a vast body of observational studies and a 
smaller body of randomized studies. I discuss each of these in turn next. Randomized 
experiments account for selection bias, but are often limited in other areas due to 
difficulty in designing and implementing proper experiments within university systems. 
Observational studies face difficulty correcting for selection bias or simply do not even 
attempt to account for it. 

B. OBSERVATIONAL STUDIES 

Koch (2006) conducted an interesting observational study at Old Dominion 
University. He utilized data from 1994 to 2001 on all courses with both a distance 
learning and resident component (p. 24). He utilizes the resulting sample of 20,428 
observations to perform OLS regressions of student characteristics on individual course 
grades (Koch, 2006, p. 25). Koch’s (2006) findings indicate that distance learning has a 
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very small impact, if any, but demographic characteristics do significantly impact student 
grades (p.26-28). The incredibly large sample size and diverse set of control variables 
represent this study’s greatest strengths. However, these strengths may not be enough to 
balance the potential biases on account of students who selected into the residence and 
distance learning courses. That said, the study highlights the need to include key 
demographic variables when analyzing the impact of distance education. 

Brown and Liedholm (2002) perform another observational study, but take a more 
focused approach than Koch (2006). They attempt to compare face-to-face classroom, 
hybrid, and virtual—their term for completely online class—versions of a Principles of 
Microeconomics class. All three versions of the class were designed to be very similar 
and utilize similar resources. The virtual version also gained access to recorded lectures 
from the face-to-face class with an additional synchronous text based component (Brown 
& Liedholm, 2002, p. 444). Brown and Liedholm (2002) find that students in the virtual 
version of class consistently score approximately half a letter grade below students in the 
face-to-face and hybrid classes (p. 447). Interestingly enough, they designed the study 
and acquired demographic data as a control, but they made no attempt to correct for 
selection bias or control for subtle, yet controllable, difference between class versions. 
For example, different instructors taught the face-to-face and hybrid classes (Brown & 
Liedholm, 2002, p. 444). Another issue exists in the small and disproportionate sample 
size. With only 363 face-to-face, 258 hybrid, and 89 virtual student observations, the 
study lacks substantial power (Brown & Liedholm, 2002, p. 445). 

Gratton-Lavoie and Stanley (2009) conduct a study with a slightly different 
methodology. They observe eight sections of an introductory economics class over four 
semesters to compare online and hybrid delivery methods (p. 7). Using summary 
statistics of final exam scores, Gratton-Lavoie and Stanley (2009) conclude that online 
education has a positive effect on learning outcomes (p. 12). However, this method is 
problematic because it neglects to consider potential impacts other than delivery mode on 
exam score. They next use a probit model to estimate the likelihood of selecting online 
over hybrid. From these likelihood values, they further assert that self-selection bias 
shifts the positive effect found earlier to an insignificant or negative one, which they do 
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not quantify (Gratton-Lavoie & Stanley, 2009, p. 20). Despite their attempt to control for 
selection, their small sample of 149 students lacks sufficient power to estimate the effects 
of online and hybrid delivery (p. 12). Also, as mentioned above, utilizing summary 
statistics to compare average final exam scores between online and hybrid courses 
presents serious limitations as a basis for determining the overall effect of online 
education on learning outcomes. They should utilize regression analysis when 
determining effect size to control for additional factors that may impact learning 
outcomes, such as previous academic performance or demographic information. 

Coates, Humphreys, Kane, and Vachris (2004) perform another observational 
study but with a heavy focus on correcting for any self-selection. They obtain data from 
three separate universities for introductory microeconomics and macroeconomics courses 
(Coates et ah, 2004, p. 535). Coates et al. (2004) utilize three separate models—OLS, 
2SLS, and an endogenous switching equation—in an attempt to ascertain the influence of 
selection bias on Test of Understanding College Economics (TUCE) scores between 
online and face-to-face students. They find that selection bias presents a substantial effect 
and that the direction of the bias points toward zero (Coates et al., 2004, p. 545). The 
presence of effect estimates biased toward zero when ignoring selection sheds additional 
light on the multitude of observational studies with findings of “no significant difference” 
between online and face-to-face modalities (Coates et al. 2004, p. 545). Also of note, 
Coates et al. (2004) present one of the few studies to address systematic differences 
between students self-selecting online courses rather than face-to-face offerings. They 
find that students selecting online or distance options are overwhelmingly employed full 
or part time in addition to school, have 300 point lower SAT scores on average, and tend 
to perform better at online courses compared to students selecting face-to-face courses 
(Coates et al., 2004, p. 545). Despite the lack of any real tangible effect, this study fills a 
crucial gap in the literature by addressing the importance of handling selection bias when 
comparing student performance between online and face-to-face courses. However, as 
with many studies on the topic, Coates et al. (2004) have only a very small sample. 
Samples for the various models used range from only 59 to 126 observations (p. 537- 
544). 
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Olitsky and Cosgrove (2012) perform a unique observational study attempting to 
account for self-selection bias. They compare multiple sections of Principles of 
Microeconomics and Principles of Macroeconomics courses, designing courses with a 
great deal of control between blended and face-to-face versions (Olitsky & Cosgrove, 
2012, p. 19). Utilizing exam scores as outcome variables, they also find no significant 
difference in effects between blended and face-to-face versions (Olitsky & Cosgrove, 
2012, p. 30). This study stands out for its method of correcting for self-selection bias. 
Olitsky and Cosgrove (2012) determine the average treatment effect of blended learning 
by utilizing propensity score matching to create a matched sample, mitigating the 
potential for selection bias (p. 27). It is this same method that we employ in our study to 
correct for selection bias. As such, Olitsky and Cosgrove (2012) provide a clear 
precedent for the use of propensity score matching to correct for self-selection bias when 
unable to randomly assign students between treatment and control groups. 

However, despite such methods of correction for selection bias among 
observational experiments, random assignment of students provides the only true means 
of completely eliminating such bias. Consequently, the following randomized 
experiments represent some of the best attempts to determine the effect of distance 
education within the current field of study. 

C. RANDOMIZED STUDIES 

Harmon, Alpert, and Lambrinos (2014) design an experiment to emulate random 
assignment of students. They randomly divide a Principles of Economics class into 
various portions of online or face-to-face delivery based on chapters. They then compute 
the likelihood of answering midterm and final exam questions correctly based on whether 
the associated questions correspond to online or face-to-face portions of the course using 
a logit model (Harmon et ah, 2014, p. 116-118). Harmon et al. (2014) reiterate the “no 
significant difference” (p. 118) findings of several other studies, yet their approach 
remains unique. However, this uniqueness coupled with a very small sample of 36 
students leads to several issues. The article lacks clarity on many details and leaves the 
reader unsure as to the validity of the approach. For example, Harmon et al. (2014) make 
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no mention of the number of sections, and based on sample size a single section is likely. 
This implies that particular exam questions were only either observed as online or face- 
to-face when computing estimates. The resulting lack of control among outcome 
variables suggests the potential for serious bias. Harmon et al. (2014) may eliminate self¬ 
selection bias from their sample, but shortcomings elsewhere leave much to be desired. 

Bowen, Chingos, Lack, and Nygren (2014) design a randomized experiment 
spanning six different universities. They compare traditional and hybrid formats for a 
statistics course and conclude that hybrid formats offer a “no-harm-done” alternative that 
also results in less time—^both for instruction and completion of deliverables—for 
students (Bowen et ah, 2014, p. 107). This otherwise well designed experiment does 
suffer from a lack of control among instructors, which is openly presented to the reader 
(Bowen et ah, 2014, p. 101). Also, this study uses the largest sample among randomized 
experiments with 605 participants (Bowen et ah, 2014, p. 98). 

Perhaps the most widely cited study within the field, Figlio, Rush, and Yin (2010) 
conduct a randomized experiment comparing exam scores between online and live 
versions of a Principles of Microeconomics course (p. 766). Figlio et al. (2010) offer 
students extra credit points in exchange for participating in an experiment. They then 
randomly assign the 327 students agreeing to participate to either live or online versions 
of the class (Figlio et ah, 2010, p. 767). Figlio et al. (2010) find no statistical difference in 
outcomes between online and live versions of the class (p. 779). Also of note, the nature 
of the experiment makes comparison to distance learning difficult. For example, students 
in the online section still had access to instructor office hours and could schedule 
individual face-to-face meetings with the instructor (Figlio et al., 2010, p. 766). 

Joyce, Crockett, Jaeger, Altindag, and O’Connell (2014) conduct another 

randomized experiment comparing traditional and hybrid formats. They also offer extra 

credit points in exchange for participation. They observe the 656 participants spread 

through eight sections of a Principles of Microeconomics course, and outcome variables 

include midterm and final exams, Applia coursework, course final grades, and 

withdrawals (Joyce et al., 2014, p. 6-9). Joyce et al. (2014) find that “traditional does 

moderately better” (p. 27), and on average students in traditional versions perform 
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2.5 percentage points higher on exams than those in hybrid formats (p. 28). Despite the 
seemingly common small sample size, this study performs exceptionally well. Joyce et al. 
(2014) manage to control for a multitude of factors such as minimizing differences 
between classes and resources and including a large set of demographic and academic 
performance variables. 

Alpert, Couch, and Harmon (2015) perform one of the best randomized studies so 
far. They randomly assign participants, who again were offered extra credit for 
participation, into either face-to-face, online, or hybrid sections of a microeconomics 
principles class between Fall of 2012 and Spring of 2014 (Alpert et al., 2015, p. 3-4). 
Alpert et al. (2015) utilize three stages of OLS regressions—each stage increasing the 
number of controls—to determine the effects of the three class formats on cumulative 
final exams scores (p. 4). They find that blended format never yields a significant effect, 
but online classes result in a significant and consistent decrease of approximately half a 
letter grade (Alpert et al., 2015, p. 27). The only concern with this study again comes in 
the form of a relatively small sample size, but this seems to be a pervasive issue among 
randomized experiments. 

The existing literature presents several relevant methodologies as well as many 
concerns. No studies exist addressing the issue of online or distance education compared 
to resident education for military officer students. In designing my study I attempt to 
apply the lessons presented above to a sample of military officer students at NFS. 
Unfortunately, I am unable to achieve the gold standard of conducting a randomized 
experiment. However, I instead focus on a methodology derived from both Koch (2006) 
and Olitsky and Cosgrove (2012). Because Koch (2006) found a large impact of 
demographics on performance outcomes, I focus on a large sample with robust 
demographic controls. Also, I utilize propensity score matching, similar to Olitsky and 
Cosgrove (2012), to mitigate biases resulting from students self-selecting into either 
distance or resident delivery modes. 


12 



III. DATA/METHODOLOGY 


Utilizing several of the strengths spread across studies within the existing 
literature, I design an observational study to estimate the impact of DL on learning and 
military performance outcomes. I merge data from three separate sources—the Naval 
Postgraduate School (NPS), the Defense Manpower Data Center (DMDC), and the 
Integrated Postsecondary Education Data System (IPEDS)—to create a sample 
containing the necessary demographic controls. I then use propensity score matching to 
correct for bias resulting from selection into either DE or residency programs. Einally, I 
conduct regression analysis to determine the impact of DE on various academic and 
military performance related dependent variables. This section addresses each of these 
data sources and methodologies in turn. 

A. NPS DATA 

1. Sample 

The primary data come from a Python extract provided by the Institutional 
Research, Reporting, and Analysis Office at NPS. These data were used in a master’s 
thesis by Kyle Alcock at NPS in March 2015, but that study focused only on a sample of 
Naval officers. The subjects in my study initially consist of the population of 10,882 NPS 
students—U.S. military officers, civilians, and international students—who began 
academic programs between the 2006 and 2013 academic years. Civilian and 
international students are excluded from the analysis sample, as I cannot match DMDC 
information to them. After dropping all civilian and international students the sample size 
decreases to 6,754 observations. 

2. Independent Variables 

a. Treatment Indicator 

The NPS data provide the binary variable for DE, where DE equals one for 
students enrolled in NPS DE degree programs. DE is the indicator of treatment for this 
study. 
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b. NPS Institutional Controls 

I generate cohort control variables from academic start year and quarter 
information. Also, school indicators allow control for the four different schools at NPS: 
the Graduate Schools of Business and Public Policy (GSBPP), the Graduate School of 
Engineering and Applied Sciences (GSEAS), the Graduate School of Operational and 
Information Sciences (GSOIS), and the School of International Graduate Studies (SIGS). 

c. Academic Preparation 

NPS utilizes academic profile codes (APC) as the primary means for determining 
academic eligibility and student academic preparation prior to admission at NPS. A 
student’s APC consists of three discrete digits, each representing a different aspect of his 
or her academic background. The first digit represents undergraduate academic 
performance based on GPA, the second digit represents mathematics background and/or 
elapsed time since college level math course completion, and the third digit represents 
engineering, science, or technical background (Naval Postgraduate School, 2016). To 
control for student preparation I generate dummy variables for each digit of APC 
indicating whether each student has met the requisite APC for his or her particular 
curriculum. 

Also, the NPS data include information on both undergraduate school name and 
time elapsed from completion of an undergraduate degree prior to beginning studies at 
NPS. Undergraduate school name allows for the merger with IPEDS data from the 
National Center of Education Statistics (NCES) that indicate the sector—private vs. 
public and for-profit vs. not-for-profit—of the college of university the military officer 
graduated from. In addition, I use undergraduate school name to generate an indicator of 
whether the student attended a service academy. With a completely military sample, 
attending a service academy also represents a level of academic preparation. Einally, 
because there is variation in elapsed time between undergraduate and graduate educations 
among NPS DE and resident students, the inclusion of the continuous variables for years 
since undergraduate education seems relevant as yet another measure of academic 
preparation. 
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d. Service and Community 

The data provide information on NFS students’ branch of military service as well 
as designator. Designator or Military Occupational Specialty (MOS) represents the 
particular job a service member performs. From this information, I generate several 
community variables and their respective interactions with service branch. Table 1 
displays the communities and respective designator/MOS by service. I generate dummy 
variables for each service, community, and the interactions of each service and 
community as controls in the analysis performed in Chapter IV. 

Table 1. Service and community breakdown 



Navy 






Army 



Air Force 



Job 

Designator 

Approx, n 



Job 

MOS 

Aoorox. r 

Job 

MOS 

Approx, n 


814 




0 



0 1 

Surface 

Surface 

1110 

786 










Surf. Undes. 

1160 

28 












0 

0 1 

Submarine 

Submarine 

1120 

302 











Sub. Undes. 

1170 

45 













47 

162 


General 

1300 

10 

Pilot 

75xx 

146 

Aviation 

ISxx 

47 

Pilot 

llxx 

126 

Aviation 

Pilot 

1310 

729 

NFO 

7525/7528 

33 




Combat 5ys. Off. 

12xx 

36 


Pilot Undes. 

1390 

21 











NFO 

1320 

279 










NFO Undes. 

1370 

3 












1 1 

88 


Oceanography 

1800 

93 

MAGTF 

02xx 

29 

Intel 

35xx 

71 

Intel 

14xx 

72 


IWO 

1810 

83 

Ground 

02xx 

7 




Cyber 

17xx 

16 

Intelligence 

IPO 

1820 

66 

CI/HUM)NT 

02xx 

4 








Intel 

1830 

75 

SI/EWO 

02xx 

12 











Air 

02xx 

14 









CI/HUMINTOPS 

02xx 

3 







_ 


1 _ £53 _ 1 

1 0 1 


Spec. War 

1130 

46 

Infantry 

03xx 

51 

Infantry 

llxx 

100 




, . 

EOD 

1140 

29 

Tank Off. 

18xx 

4 

Armor 

19xx 

2 




orouno Comoat 

£00 Undes. 

1190 

1 

AAV Off. 

18xx 

2 

Field Artillery 

13xx 

87 








Field Artillery Off. 

08xx 

38 

A/D Artillery 

14xx 

17 








Special Forces 

18xx 

247 







1 _ 294 _1 

1 502 


EDO 

1440 

11 

Adjutant 

Olxx 

39 

Imm. 8i Personnel 

OOxx 

29 

Weather 

ISxx 

71 


EDO Undes. 

1460 

281 

Logistics Off. 

04xx 

81 

Corps, of Eng. 

12xx 

68 

Ops. Support 

16xx 

42 


AEDO Aero 

1510 

44 

Comms. Off. 

06xx 

111 

Signal Corps. 

25xx 

52 

Security Forces 

31xx 

19 


AEDO Maint 

1520 

78 

Combat Eng. Off. 

13xx 

25 

MP 

31xx 

7 

Civil Engineering 

32xx 

16 


PAO 

1600 

174 

Ground Supp. Off. 

30xx 

66 

Psy. Ops. 

37/39XX 

5 

Public Affairs 

35xx 

10 


FAO 

1710 

13 

Finance Off. 

34xx 

32 

Civil Affairs 

38xx 

27 

Personnel 

38xx 

12 


FAO Undes. 

1720 

42 

PAO 

43xx 

3 

Space Ops. 

40xx 

1 

Biomedical Sp 

43xx 

2 


HR 

1200 

189 

JAG 

44zz 

6 

Adjutant Gen. 

42xx 

10 

Scientific Research 

61xx 

11 


Medical 

2100 

10 

MP 

58xx 

5 

Finance 

44x 

7 

Developmental Eng. 

62xx 

44 


Dental 

2200 

3 

A/C Maint. Off. 

60xx 

22 

Foreign Affairs 

48xx 

8 

Acquisitions 

63xx 

61 


Medical Service 

2300 

65 

Avioincs Off. 

63xx 

1 

Acquisitions Off. 

51xx 

8 

Contracting 

64xx 

79 


Flight Surgeon 

2302 

1 

Avn. Supply Off. 

66xx 

27 

Medical 

60xx 

1 

Finance 

65xx 

24 

Support 

JAG 

2500 

7 

AirC&C Off. 

72xx 

9 

Medical Service 

67xx 

9 

Special Inv. 

71xx 

53 

Nurse 

2900 

12 

Low Alt. A/D 

72xx 

9 

T ransportation/Logistics 

88xx 

14 

A/C Maint. 

21xx 

17 


Supply 

3100 

332 

Air Supp. Control 

72xx 

11 

Ammunition/Maint/Ord. 

91xx 

20 

Missile Maint. 

21xx 

2 


Chaplain 

4100 

2 

A/D Control 

72xx 

8 

Quartermaster 

92xx 

16 

Readiness 

21xx 

24 


Civil Engineering Corps 

5100 

20 

ATC 

72xx 

10 

O.R./Sys Analysis 

49/57XX 

4 

Spec. Duty 

88xx 

2 


Spec. WarLDO 

6152 

1 

Avn. Acquisitions 

80xx 

1 

Chemical 

74xx 

8 

Students/candidates 

92xx 

13 


Elec. Surf. LDO 

6180 

1 

Defense Systems Analyst 

88xx 

1 







Eng. Sub LDO 

6232 

1 









Elec. Sub LDO 

6280 

1 









Comm. Sub LDO 

6290 

1 










Avn. Maint. LDO 

6330 

6 










Air Traffic Control LDO 

6390 

1 











Admin LDO 

6410 

5 











IS LDO 

6420 

2 










Met/Ocean LDO 

6460 

1 
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3. Dependent Variables 

The NFS data provide three of the five dependent variables that I analyze. Total 
quality point rating (TQPR) is essentially a GPA calculated for all courses taken at NFS 
(Naval Postgraduate School, 2015). I use TQPR as the primary and only continuous 
outcome variable. It would be advantageous to utilize individual course grades instead of 
overall TQPR, but this information is currently unavailable for the existing sample. I also 
include binary outcome of whether a student graduates. 

B. DMDC DATA 

1. Sample 

As Koch (2006) makes clear, sound demographic control variables are necessary 
when ascertaining the impact of DL. DMDC data provide these important demographic 
controls for the sample, as well as subsequent career progression of these students upon 
leaving NFS. Demographic data were requested for the entire sample as of six months 
prior to beginning their studies at NFS, while work performance data was requested 
covering the period after leaving NFS. Thus, the sample size remains the same, and forty- 
seven additional observables are added. 

2. Independent Variables 

Of the additional observable characteristics added, I focus on key demographic 
variables including: gender, race, marital status, age, number of dependents, and rank. 
Race consists of variables for white, black, Hispanic, and other race. Marital status 
includes two dummy variables, one indicating whether an observation was married 
during his or her time at NFS and the other indicates if the observations had ever 
experienced a divorce. Rank indicates the military paygrade of an observation upon 
beginning enrollment at NPS. The fourteen resulting variables cover the range of 
demographics typically controlled for. 

3. Dependent Variables 

DMDC data provide the remaining two outcome variables in this study, which are 

promotion and separation status. The dependent variable “promoted” indicates students 
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who received a promotion after departure from NPS, whether or not they had actually 
received a degree. Separated indicates those officers that separated from military service 
for any reason after leaving NPS. 

C. IPEDS DATA 

1. Sample 

IPEDS is a branch of the National Center for Education Statistics (NCES) and 
provides downloadable and publically available data for accredited schools offering 
postsecondary education within the United States (National Center for Education 
Statistics, 2016a). I merge the 2012 IPEDS database, which is the most applicable, to the 
existing sample based on undergraduate degree institution listed in the NPS data. 

2. Variables 

Unlike the several variables DMDC data provide, I only use IPEDS for the 
addition of the single independent variable “sector.” The IPEDS sector represents a scale 
of one through nine accounting for both control and level of an institution. Control 
represents whether the school is public, private not-for-profit, or private for-profit. The 
level indicates whether a school offers four, two, or less-than-two-year degrees (National 
Center for Education Statistics, 2016b). Table 2 illustrates the various values for sector as 
provided by IPEDS (National Center for Education Statistics, 2016c). 


Table 2. IPEDS sector breakdown 


Code 

Definition 

1 

Public, 4-year or above 

2 

Private not-for-profit, 4-year or above 

3 

Private for-profit, 4-year or above 

4 

Public, 2-year 

5 

Private not-for-profit, 2-year 

6 

Private for-profit, 2-year 

7 

Public, less than 2-year 

8 

Private not-for-profit, less than 2-year 

9 

Private for-profit, less than 2-year 
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However, I focus solely on the control classification because commissioned 
officers overwhelmingly possess four year degrees. As such, I generate three control 
variables for the entire sample: public, private not-for-profit, and private for-profit. 
Additionally, I define these three variables so that they are exclusive of Service Academy 
graduates. Service Academies fall under public category according to IPEDS, however 
based on selectivity and average student performance prior to postsecondary education 
they are more akin to private not-for-profit institutions. For this reason, and because a 
large proportion of NPS students are Service Academy graduates, I utilize the variable 
“Service Academy” from the NPS data to represent a fourth component of sector. 

D. DATA SUMMARY 

The final merged sample retains the original 6,754 observations, approximately 
20 percent of which are for DL students. Table 3 presents a summary of the number of 
observations within various subgroups of the sample. 
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Table 3. Sample summary 



DL 

Resident 

All 

Full Sample 

1,331 

5,423 

6,754 

By Variable: 




^ Navy 

1,182 

2,773 

3,955 

Marine 

54 

767 

821 

^ Army 

16 

910 

926 

Air Force 

69 

887 

956 

Surface 

104 

710 

814 

Submarine 

122 

225 

347 

g Aviation 

697 

764 

1,461 

1 Intelligence 

24 

690 

714 

o 

U Ground Combat 

33 

776 

809 

Support 

337 

2,449 

2,786 

01 

29 

122 

151 

02 

46 

473 

519 

03 

706 

2,883 

3,589 

04 

272 

1,726 

1,998 

05 

130 

50 

180 

06 

14 

4 

18 

Public 

457 

2,461 

2,918 

2 Private not-for-profit 
Private for-profit 

223 

1,108 

1,331 

5 

41 

50 

Service Academy 

412 

1,230 

1,642 

GSBPP 

772 

1,150 

1,922 

8 GSEAS 

361 

1,055 

1,416 

1 GSOIS 

190 

1,697 

1,887 

SIGS 

8 

1,384 

1,392 

^ Met APC 1 

1,019 

4,863 

5,882 

^ Met APC 2 

845 

4,166 

5,011 

Met APC 3 

1,084 

4,887 

5,971 

Female 

70 

523 

593 

u White 

949 

4,021 

4,970 

Oh Black 

M Hispanic 

S Other Race 

Q Married 

64 

356 

420 

82 

375 

457 

236 

671 

907 

961 

3,963 

4,924 

Divorced 

4 

102 

106 
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There are several systematic differences between DL and resident students. 
Figures 1 through 11 illustrate the differences between DL and resident students based on 
the independent variable categories of service, community, rank, school, APC, 
undergraduate degree institution sector, gender, marital status, and race. 

A comparison of services between DL and residency are depicted in Figure 1. 
Unsurprisingly, Navy students comprise the majority among residents. However, for DL 
this majority nearly doubles. Marine Corps, Army, and Air Force students comprise only 
about 10 percent of all DL students at NFS between 2006 and 2013. 


Figure 1. Military service branch comparison for DL and residency 



Warfare communities are depicted in Figure 2. The makeup of DL and resident 
students is quite different. In particular, aviators tend to favor and/or get assigned to DL 
programs more than other communities. While aviators only make up approximately 12 
percent of all resident students between 2006 and 2013, they comprise more than half of 
the DL students for the same period. The disparities presented only reinforce the need to 
control for both service and community in assessing student outcomes, and also in order 
to make an apples-to-apples comparison. 
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Figure 2. Warfare community comparison for DL and residency 



Intelligence 

2% 


Ground 

Combat 


DL Surface 


Submarine 


Resident 



Figure 3 illustrates rank breakdowns between DL and residency. The only 
substantial differences seem to be a decrease in 0-2’s, a decrease in 0-4’s, and an 
increase in 0-5’s within DL. Figure 4 shows the increased representation of GSBPP 
within DL compared to residency, while also displaying a decreasing representation by 
GSOIS and SIGS. 

Figure 3. Military rank comparison for DL and residency 

Resident 
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Figure 4. NFS school comparison for DL and residency 



DL 

SIGS 


Resident 



Figures 5 through 7 depict whether or not students met the required APCs for 
their respective curriculums. Interestingly enough, comparing AFC results show that DL 
students tend to be far less prepared with regard to all APC’s than resident students. The 
greatest gap is in AFC 2 (mathematics background). 


Figure 5. AFC 1 undergraduate GPA comparison for DL and residency 

DL Resident 
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Figure 6. APC 2 undergraduate mathematics background comparison for DL 

and residency 


DL 


Resident 



Figure 7. APC 3 undergraduate science and technical background 
comparison for DL and residency 


DL 


Resident 




Figure 8 depicts the IPEDS sector breakdown. Sector distribution in DL is 
comparable with the distribution in residency programs, with no difference in overall 
percentage of private not-for-profit and private for-profit institutions. However, students 
with undergraduate degrees from public institutions decrease by 10 percentage points 
within DL compared to residency, while an increase in Service Academy graduates 
makes up the difference. Gender, shown in Ligure 9, also shows only a small difference 
between DL and residency. There are 4 percentage points fewer females in DL than in 
residency. 
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Figure 8. Undergraduate institution sector comparison for DL and residency 


DL 


Resident 




Figure 9. Gender comparison for DL and residency 



DL and resident military students tend to be similarly distributed with regard to 
marital status as well. Figure 10 shows that DL students represent only 1 percentage point 
more unmarried students than their resident counterparts. 
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Figure 10. Marital status comparison for DL and residency. 


DL 


Resident 



Figure 11 depicts the race breakdown between DL and residency. Race presents 
yet another similar distribution between DL and residency, with only a slight increase in 
other race within DL. 


Figure 11. Race comparison for DL and residency. 


DL 

Other/Unknown 



Resident 



Clearly, there exist systematic differences between DL and resident students at 
NFS. Students who are in the Navy and in the aviation community tend to be over¬ 
represented in DL. Also, DL students suffer from being less prepared on average, 
especially in mathematics, than resident students. These are but a few of the large 
differences between DL and residency, yet many subtle disparities also exist in the data. 
Thus, it is imperative to include all of the above categories as controls and to use them for 
propensity score matching. 
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E. METHODOLOGY 


The method of propensity score matching conceptually involves creating 
counterfactual outcomes of what would have happened to DL students had they gone 
through a resident program, and what would have happened to resident students had they 
been through a DL program. 

The matching methodology is broken down into two distinct stages. Stage One 
involves creating a matched subsample from the existing sample. Stage Two of my 
analysis then uses the matched sample to conduct regression analysis and determine the 
impact of DL on performance outcomes. I further discuss the methodology behind these 
two stages in the next two sections. 

1. Stage One 

Propensity score matching is a statistical technique that allows for the creation of 
a matched sample where observations are similar enough between treatment and control 
groups to efficiently and unbiasedly determine the average effect of the treatment. Also, 
on can determine the treatment effect on the treated using this method. Matching relies on 
the key assumption that subjects that are similar based upon observable characteristics are 
likely also similar on unobservable characteristics (Gertler, 2011). In this study, the 
variable DL represents treatment. Consequently, within a matched sample, those 
observations enrolled in DL are similar with respect to observable characteristics, and 
thus also unobservable characteristics, to those enrolled as resident students. In a sense, 
this process is simulating a nonexistent control group as a counterfactual for the purposes 
of examining the effect of DL on learning and job performance outcomes. Thus, the 
counterfactual match to a DL student is identified by finding a control resident with the 
same propensity to be in DL and vice versa. 

To create this matched sample, I start by determining the probability that each 
observation is enrolled in DL. Considering DL is a binary variable, a Logit model suits 
this purpose well. Equation 1 specifies this logit model, where the associated independent 
variables are defined in Table 4. 
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prob(jyL. = 1) = 


^*0 +b,DEMOi +b2ACADi +b,MILi +btCHRT, 

_|_ ^b„+biDEMOi+b2ACAD.+b2MIL-+b^CHRT. 


( 1 ) 


Table 4. Independent variables 


Demographic Controls (DEMO) 

Academic Controls (ACAD) 

Military Controls (MIL) 

Cohort Controls (CHRT) 

Female 

Years since Undergraduate Degree (Continuous) 

Service (Navy Reference) 

School (GSBPP Referece) 

Race (White Reference) 

Sector (Public Reference) 

Community (Support Reference) 

Cohort Year (2006 Reference) 

Married 

Met APC 1, 2, 3 

Service Community Interactions 

Cohort Quarter (Q1 Reference) 

Divorced 


Rank (0-3 Reference) 


Age (Continuous) 




Number of Dependents (Continuous) 









Upon estimating this regression, predicted values of DL are calculated for all 
observations. I then utilized a kernel density function to plot probability density overlays 
for both DL=1 and DL=0. This overlay allows for visual inspection of the region of 
common support, or the region where observations are similar in their propensity for 
treatment (Gertler, 2011). Ideally the probability distributions do not overlap perfectly, as 
the goal is to eliminate dissimilar observations at the far left and right ends of the overlay. 
Figure 12 presents an example overlay plot. 

Figure 12. Example propensity score matching overlay 



Source: Gertler, 2011, p. 110 
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To achieve a sound probability overlay requires some trial and error. I modified 
the model in Equation 1 several times to acquire a desirable overlay and satisfy the 
necessary assumption of common support. I eventually acquire a suitable region of 
common support by attempting various combinations of independent variables. I then 
create the final matched sample by eliminating observations that fall outside the region of 
common support. 

2. Stage Two 

Stage Two is simply conducting a normal regression analysis using the matched 
sample. To form unbiased and efficient estimates, I use the inverse of propensity scores 
generated from the first stage as weights in the second stage regressions. Four outcome 
variables are evaluated to ascertain the impact of DL. TQPR is the only continuous 
outcome, while Graduated, Promoted, and Separated are all binary dependent variables. 
An original least squares (OLS) regression is an obvious choice for TQPR, however the 
binary dependent variables require some thought. 

I initially specify the three binary outcomes as logit models. Upon closer 
examination, I realize that a Linear Probability Model (LPM) is the superior choice. The 
logit model derivatives, or marginal effects, for all binary outcomes are nearly identical 
to the coefficients of a similarly specified LPM. Further, the LPM’s key weakness lies it 
its ability to produce predicted values less than zero or greater than one, which are not 
feasible probability values (Woolridge, 2013). I generate predicted values for a LPM 
regression of each binary outcome and summarize the results. For each outcome, the 
predicted values fall within the acceptable range between the 5th and 95th percentiles. 
Also similar to its OLS counterpart, the LPM is the best linear unbiased estimator. Thus, 
the LPM is both qualitatively and quantitatively the superior choice of model. Equations 
2 and 3 depict the specifications for continuous and binary outcomes, respectively. 

y. =bQ+ b,DL. + b^DEMO. + b^ACAD. + b^MlL. + b^CHRT^ (2) 

prob{ y,. =l)=bQ+ bpL + b^DEMO. + b^ACAD^ + b^MIL. + b^CHRT. (3) 
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Due to the matching process, bi represents the treatment effect of DL in both 
Equations 2 and 3. Under propensity score matching assumptions, this treatment effect is 
also free of self-selection bias. 

The final step involves running several regressions to determine the heterogeneity 
of DL’s effect within different subgroups of the matched sample. I run a series of 112 
separate regressions where each service, community, rank, APC, sector, and school are 
isolated. Equations 4 and 5 depict the specifications for continuous and binary outcomes, 
respectively, where CONTROL^ indicates the subgroup being examined in isolation. 

y. =b^+ b^DEMO ,, s. t. CONTROE. = 1 (4) 

prob( y,. = 1) = + bpEMO ., s. t .CONTROE j = \ (5) 

These final models allow comparison of DE’s effect between different services, 
communities, ranks, schools, and among those students that did or did not meet the 
requisite APC for their curriculum. 


29 



THIS PAGE INTENTIONALLY LEFT BLANK 


30 



IV. RESULTS 


In this chapter, I present the results in three sections. First, I show the first stage 
results of the matching exercise. Second, I present the second stage results estimating the 
impact of DL on the four outcome variables, namely TQPR, Graduate, Promoted, and 
Separated. Finally, I test for heterogeneous effects of DL within subgroups of the 
matched sample. 

A. STAGE ONE 

As mentioned in Chapter III, to create a matched sample I first regress observable 
characteristics of students on an indicator for DL participation using logistic regressions. 
The key observable factors included in the logit model are demographic characteristics, 
military service branch, warfare community and its associated interactions with service 
branch, and military rank. The full results of this regression appear in Appendix A. It is 
important to note that gender, marital status, time elapsed since undergraduate education, 
undergraduate university sector, service branch, the support community, and support 
community and service interactions are all significant predictors of enrolling in DL. Also, 
the ranks of 0-2 and 0-3 are significant predictors of selection of DL programs as well. 

From the logistic regression, I calculate predicted values for the likelihood of DL 
enrollment and then plot probability density overlays for different combinations of 
observable characteristics. Figure 13 shows the probability density functions separately 
for resident and distance learning students. 
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Figure 13. Propensity seore matehing overlay 



Pr(dl) 

Residence Learning - Distance Learning 


The x-axis of the plot in Figure 13 represents the propensity seore. These values 
represent the predicted values resulting from the logistic regression. The region of 
common support exists between approximately 0.1 and 0.3. Observed students falling 
outside of this range are eliminated from the sample as they are significantly dissimilar to 
those within the region of common support. Doing so results in a matched sample of 
5,289 observations. This reduction of only 1,465 students from the initial sample results 
in a matched sample that is sufficiently large and robust to provide statistical power to 
my analysis. 

B. STAGE TWO 

Table 5 displays the average treatment effect (ATE) of DL for each performance 
outcome. These results are representative of the entire matched sample. Also, full 
regression results appear in Appendices B through E. 
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Table 5. Average treatment effect summary 


Outcome 

ATE ofDL 

TQPR 

-0.3998*** 

[0.0735] 

Graduate 

-0.2232*** 

[0.0272] 

Promoted 

-0.1253*** 

[0.0276] 

Separated 

0.0562*** 

[0.0203] 


Robust standard errors in brackets 
*** p<0.01, ** p<0.05, * p<0.1 


The first row of Table 5 focuses on the results for TQPR (average GPA). All else 
equal, the TQPR of students in DL programs is 0.3998 points below resident students. 
This result is statistically significant at the 1 percent level. This negative impact translates 
into almost half a letter grade reduction in a DL student’s TQPR compared to an 
equivalent resident student. 

It is also important to note that DL is not the only factor that affects TQPR. 
Appendix B displays complete regression results for this outcome. Married students on 
average have higher TQPRs than unmarried students. Marine Corps students have 
approximately a half a letter grade higher TQPR than Navy students. Surface warfare 
officers and submariners both have lower TQPRs than support officers, while Naval 
Aviators have higher TQPRs. Students in GSEAS, GSOIS, and SIGS all have lower 
TQPRs than students in GSBPP. Finally, service academy graduates have significantly 
higher TQPRs than students that graduated from public institutions for their 
undergraduate degree. Interestingly, meeting APC 3 results in a 0.2127 point reduction in 
TQPR on average. 

Based on the existing literature, it appears that NPS students enrolled in DL 

programs are significantly worse off than their civilian equivalents. These results are 

somewhat inconsistent with the common overall findings of little to no significant 

difference between traditional and online class formats. However, they are consistent 

with the findings of Bernard et al. (2004) in that synchronous distance programs perform 
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worse than asynchronous equivalents. The NFS DL programs being synchronous support 
the general finding of lower performance in the synchronous format compared to 
asynchronous. 

The second row of Table 5 shows the results for graduation. All else being equal, 
I find that DL results in a decrease in the probability of graduating by 22.32 percentage 
points on average. This negative impact is significant at the 1 percent level. This result 
implies that even if resident programs achieved a 100 percent graduation rate, that DL 
equivalents would suffer more than one-fifth of their students failing to complete 
programs of study. Again, synchronous DL at NFS seems more detrimental than existing 
literature would suggest. 

Similar to TQFR, other factors significantly affect the chances of graduating from 
NFS. For example. Appendix C shows that on average black students are 11 percentage 
points less likely to graduate than white students. Also like TQFR, being a surface 
warfare officer or a submariner results in a negative impact on the probability of 
graduation. Marine Corps support officers and Air Force support officers are also less 
likely to graduate. Further, 0-5s are less likely to graduate than 0-3s and students in 
SIGS are not significantly less likely to graduate than GSBFF students. Again, meeting 
AFC 3 has a negative impact on the probability of graduation. However, AFC 2 shows a 
7 percentage point increase in the probability of graduating. 

The third row of Table 5 suggests DL students are less likely to promote after 
NFS. The coefficient on DL indicates they are less likely to promote by 12.53 percentage 
points on average. This negative impact is significant at the 1 percent level. Considering 
that many military officers view graduate education as a means of increasing their chance 
of promotion, this particular effect of DL is noteworthy. 

Unlike TQFR and graduation, the impact of DL on promotion is smaller than 
several other effects. Appendix D shows that NFS Marine Corps officers are 
20 percentage points more likely to promote than Navy officers at NFS. Surface warfare 
officers and Aviators are significantly less likely to promote than support officers. Also, 
0-4s, 0-5s, and 0-6s are all significantly less likely to promote after NFS than 0-3s. 
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The fourth row of Table 5 shows the impact of DL on separation. All else equal, I 
find that DL results in an increase in the probability of separating from the military by 
5.62 percentage points on average. This result is significant at the 1 percent level. 
Although a positive value, separation represents a negative outcome, thus DL provides 
yet another negative impact on performance at NFS. Although the percentage is small, 
this negative impact of DL suggests that the military would have a more difficult time 
obtaining a return on investment for DL graduates. 

Appendix E shows the other factors affecting the probability of separating from 
active service. Similar to promotion results, DL is one of the smaller significant effects 
on separation. Being Hispanic results in a 6 percentage point decrease in the probability 
of separating after NFS, while being married results in an 11 percentage point decrease in 
the probability of separation after NFS. Marine Corps officers are also 11 percentage 
points less likely to separate than Navy officers. Students graduating from Frivate for- 
profit institutions for their undergraduate degrees are also 12 percentage points less likely 
to separate. Although DL is small in magnitude, it is one of the only characteristics that 
increase the likelihood of separation after NFS. 

C. HETEROGENEITY 

An important finding across several studies in the literature is the presence of 
heterogeneous effects of distance or online instruction on student characteristics. In 
section B, I estimate an average treatment effect for all types of students. In this section, I 
split the sample into various subgroups to test for treatment heterogeneity. Specifically I 
split the sample by service branch, warfare community, rank, AFC, sector, and school in 
the following sections. Each of the following sections represents groupings of these 
mutually exclusive subgroups. Eor example, individual services are mutually exclusive, 
as a student cannot be in both the Navy and Army simultaneously. A student is either in 
the Navy, the Army, The Marine Corps, or the Air Eorce. 

Appendices E through I show full regression results for the outcomes TQFR, 
Graduate, Fromoted, and Separated, respectively. Within these appended tables each row 
represents an individual regression for the sample subgroup listed in the first column. 
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Tables 6 through 11 compile the coefficients for DL within each of the four appended 
tables of full results. Also, all tables within this section highlight sample subgroups in red 
to represent small sample sizes of less than 100 students enrolled in DL. These 
highlighted subgroups indicate results that are considered insignificant as they lack 
sufficient statistical power to determine the effect of DL. 

1. Service 

Table 6 represents the treatment effect of DL within each of the four services 
present within the matched sample. Within the Marine Corps and Army no significant 
difference exists between DL and resident students with respect to TQPR. However, 
Marine DL students tend to have lower probabilities of graduating than Marine residents. 
Army DL students tend to have lower graduation rates, lower promotion rates, and higher 
separation rates than residents. 


Table 6. Treatment heterogeneity by service 


ATE ofDL 



TQPR 

Graduate=l 

Promoted 

Separated 

n 

DL=1 

Navy 

-0.1647*** 

-0.0778*** 

0.0477** 

-0.0020 

3,055 

776 

MarineCorps 

-0.2616 

-0.2122** 

-0.0792 

0.0410 

684 

41 

Army 

-0.8085 

-0.2611 

-0.3940*** 

0.2113*** 

755 

7 

AirForce 

-0.1833*** 

-0.1644*** 

-0.3997*** 

0.2012*** 

754 

55 


Navy DL students have significantly decreased TQPRs, a lower probability of 
graduation, and a higher probability of promotion after NPS than resident Navy students. 
DL has no significant effect on separation within the Navy as a whole. Marine Corps, 
Army, and Air Force students have no significant effect due to insufficient sample size. 

2. Community 

Table 7 represents the treatment effect of DL within different warfare 
communities. Due to sample size limitations, only the aviation and support communities 
are considered significant. Aviators, a numerically important group, show smaller 
negative effects of DL on TQPR with no significant impact of DL on the probability of 
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promotion or separation. DL students within the support community show similar effects 
of DL to the overall sample. The exception is for separation, as students in the support 
community show no significant impact of DL on separation from active service. 


Table 7. Treatment heterogeneity by community 


ATE ofDL 



TQPR 

Graduate=l 

Promoted 

Separated 

n 

DE=1 

Surface 

-0.5322*** 

-0.2837*** 

-0.2226*** 

-0.0430 

695 

68 

Submarine 

-1.2189*** 

-0.5338*** 

-0.2869*** 

-0.0749*** 

230 

51 

Aviation 

-0.0646*** 

-0.0014 

-0.0361 

0.0315 

1,164 

512 

Intelligence 

-0.4580 

-0.1277 

-0.1902*** 

0.3359* 

590 

13 

GroundCombc 

-0.4520** 

-0.2332** 

-0.2917*** 

0.3681*** 

684 

18 

Support 

-0.3375** 

-0.1997*** 

-0.2024*** 

0.0452 

2,131 

207 


3. Rank 

The varying treatment effects of DL with respect to individual ranks are shown in 
Table 8. 0-3s and 0-5s are the only ranks with a significant negative impact of DL on 
TQPR, with 0-5’s showing slightly more negative results. With respect to graduation, O- 
5s again see the largest decrease in probability for DL. 0-4s are the only rank to see a 
significant impact on promotion between DL and residency, which negative and almost 
twice as much so as the full sample average. Finally, DL 0-3s are significantly more 
likely to separate than their resident counterparts. 


Table 8. Treatment heterogeneity by rank 


ATE ofDE 



TQPR 

Graduate=l 

Promoted 

Separated 

n 

DL=1 

01 

-1.0511 

-0.2817 

-0.2514 

-0.0078 

123 

13 

02 

-0.1901* 

-0.0873 

0.0038 

0.1515** 

438 

29 

03 

-0.1430*** 

-0.1014*** 

-0.0469 

0.0605** 

2,950 

530 

04 

-0.3173* 

-0.1545*** 

-0.2406*** 

0.0570 

1,604 

198 

05 

-0.2104** 

-0.3534*** 

0.0207 

-0.1503* 

139 

100 

06 

-0.1747 

-0.4017 

-0.0920 

0.0000 

11 

9 
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4. 


APC 


Table 9 illustrates the varying effects of treatment for DL with respect to 
academic preparation. There is a clear and significant difference in the impact of DL 
between students that did and did not meet each individual APC category. 


Table 9. Treatment heterogeneity by APC 


ATE ofDL 



TQPR 

Graduate=l 

Promoted 

Separated 

n 

DL=1 

met_apcl 

-0.3015*** 

-0.1619*** 

-0.1086*** 

0.0462** 

611 

185 

met_apc2 

-0.2574*** 

-0.1200*** 

-0.1088*** 

0.0560** 

1,271 

278 

met_apc3 

-0.2961*** 

-0.1558*** 

-0.1101*** 

0.0486** 

551 

140 

notmet_apcl 

-0.0265 

-0.0695* 

-0.1202** 

0.1663*** 

4,768 

694 

notmet_apc2 

-0.1501 

-0.0813* 

-0.0416 

0.1265*** 

4,018 

601 

notmet apc3 

0.0199 

-0.0360 

-0.0655 

0.1468*** 

4,738 

739 


These results are inconsistent with existing literature in that stronger or more 
prepared students tend to perform worse in DL. For students who meet APC 
requirements, DL has a significant and negative impact on all four outcomes. However, 
students who do not meet the requisite APC show no significant difference between DL 
and residency. These results are increasingly relevant when considering the data 
breakdowns presented in Chapter III. Not only do DL students not meeting requisite APC 
perform no worse than resident students, but the percentage of students not meeting APC 
within DL is roughly twice that of within resident programs. 

5. Sector 

The type of institution students receive their undergraduate degrees presents more 
interesting information. Table 10 shows the treatment effect of DL for the four sectors of 
undergraduate institutions within this study. Service Academy graduates display no 
significant effect of DL. Further, students who graduated from public civilian universities 
for their undergraduate degree experience a significant and more negative impact of DL 
than students graduating from private not-for-profit institutions. 
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Table 10. Treatment heterogeneity by sector 


ATEofDL 



TQPR 

Graduate=l 

Promoted 

Separated 

n 

DL=1 

Public 

-0.3541*** 

-0.1565*** 

-0.0880** 

0.0695** 

2,679 

363 

PrivateNEP 

-0.1921*** 

-0.1524*** 

-0.0666 

0.0418 

1,123 

186 

PrivateEP 

0.1093 

-0.1509 

-0.1796 

-0.0101 

43 

4 

Service Acade 

-0.0727* 

-0.0591* 

-0.1456*** 

0.1432*** 

1,435 

324 


It is also interesting to see that Service Academy graduates experience the greatest 
decrease in probability of promotion and the largest increase in probability of separation 
as a result of DL. Students from public civilian institutions have a similar impact of DL 
on promotion and separation; however, the magnitude is roughly half that of Service 
Academy graduates. 

6. School 

Schools within NFS also present some interesting heterogeneity. Table 11 shows 
treatment effects of DL for the four schools. GSEAS and GSOIS show similarly negative 
impacts for DL with respect to both TQPR and probability of graduation. They differ 
with respect to late graduation, however. DL actually has a significant and slightly 
positive effect on the probability of graduating on time within GSOIS. More relevant, 
however, is that students within GSBPP show no significant difference between DL and 
residency on all outcomes except the probability of graduation. 


Table 11. Treatment heterogeneity by school 


ATEofDL 



TQPR 

Graduate=l 

Promoted 

Separated 

n 

DL=1 

GSBPP 

-0.0982 

-0.1114*** 

-0.0424 

0.0264 

1,477 

555 

GSEAS 

-0.2147*** 

-0.1107*** 

-0.0508 

0.0760* 

1,099 

203 

GSOIS 

-0.7181*** 

-0.2646*** 

-0.1691*** 

0.0437 

1,509 

119 

SIGS 

0.0981** 

0.2404*** 

-0.4227*** 

0.7918*** 

1,086 

2 


39 






















THIS PAGE INTENTIONALLY LEFT BLANK 


40 



V. SUMMARY AND CONCLUSIONS 


A. SUMMARY 

I merge data from three separate sources to create a robust sample of NPS military 
officer students. These data contain sound and diverse demographic and undergraduate 
institution information. Using these diverse controls and employing propensity score 
matching, I mitigate the pervasive issues present within the existing literature addressing 
observational studies, small sample sizes, and demographic controls. 

Utilizing the matched sample I find that DL has a significant and negative impact 
on both student academic performance and subsequent job performance. Based on the 
findings of existing literature, NPS students seem to be worse off than civilian 
equivalents. Further, results are heterogeneous within different military services, warfare 
communities, and levels of academic preparation. 

B. RECOMMENDATIONS 

When looking at the results, several potential means of improving the impact of 
DL come to mind. First, focusing on programs or students who are more likely to 
perform well in DL could help mitigate the existing negative impact on performance. For 
example, students within GSBPP show no significant difference in performance between 
DL and residency. With regard to warfare communities. Aviators suffer no significant 
impact of DL as well. However, it is worth noting that several of these student 
characteristics that tend to favor DL are already heavily represented within the existing 
DL population. More than half of the students within the sample are already enrolled in 
programs within GSBPP. Also, more than half of all DL students between 2006 and 2013 
are Aviators. Thus, due to the existing makeup of DL, this solution may not be entirely 
feasible. 

Another area of potential improvement revolves around APC requirements. I 

recommend creating less stringent requirements for admission to DL, or simply 

continuing to waive existing requirements. Chapter III clearly demonstrated the large 

number of students within DL who do not meet requisite APC values for their 
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curriculums. This observation is even more relevant when coupled with the 
heterogeneous findings that students who do meet requisite APC suffer a significantly 
negative impact of DL while those that do not meet requisite APC show no significant 
effect of DL. Waiving APC requirements within DL does not seem to be manifesting any 
significantly negative effect on students’ grades. 

Next, I recommend shifting DL at NPS to a more asynchronous format. Although 
there are asynchronous aspects of classes within NPS DL programs, they are primarily 
synchronous in their delivery. The existing literature clearly addresses the superiority of 
asynchronous formats over synchronous formats with regard to DL. Further, as most of 
the studies that find no significant difference between distance and traditional formats are 
based on asynchronous online programs, this difference in format could help explain the 
uncharacteristically negative impact of DL at NPS. 

Finally, I recommend conducting additional research on the impact of DL at NPS. 
Particularly, the ability to control for individual course grades seems a logical next step. 
The majority of existing literature focusses on individual courses and their associated 
course grades when determining the impact of DL. As such, utilizing similar performance 
outcomes could provide a better comparison to civilian equivalents. 
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APPENDIX A. PROPENSITY SCORE MATCHING-LOGIT 

RESULTS 


Matching Logit - DL 

VARIABLE COEFFICIENT 


Female 


-0.3603** 



[0.1710] 

Black 


-0.2683 

[0.1754] 

Hispanic 


-0.0043 

[0.1636] 

OtherRace 


-0.0314 

[0.1461] 

Manied 


0.5472*** 



[0.1311] 

Divorced 


0.0837 

[0.5625] 

Age 


-0.0051 



[0.0148] 

YrsFrUGrad 


0.0750*** 



[0.0153] 

DEPS 


-0.0543 



[0.0369] 

sector 


-0.1992*** 

[0.0587] 

MarineCoips 


-1.5558*** 

[0.2047] 

Army 


-4.4654*** 

[0.5668] 

AirForce 


-2.4622*** 



[0.3605] 

Support 


1.1198*** 

[0.3806] 

Navy_Support 


-2.0320*** 



[0.3959] 

Marine_Support 


-1.8838*** 

[0.5066] 

Army_Support 


-0.2678 

[0.8399] 

01 


-0.1529 

[0.3314] 

02 


-0.7505*** 

[0.2095] 

04 


-0.1091 

[0.1261] 

05 


1.8117*** 



[0.2802] 

06 


1.8150** 

[0.9055] 

Constant 


-1.3519*** 

[0.3987] 


Observations 


5,290 

Robust standard errors in brackets 

*** p<0.01. 

** p<0.05, 

*p<0.1 
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APPENDIX B. OLS REGRESSION RESULTS FOR THE OUTCOME 

TQPR 


_TQPR_ 

VARIABLE COEFFICIENT VARIABLE COEFHCIENT 


dl 

-0.3998*** 

M arine_S upport 

-0.6372*** 


[0.0735] 


[0.2242] 

Female 

-0.0879 

Army Support 

-0.4874 


[0.0832] 


[0.3655] 

Black 

-0.1185 

AirForce_Support 

-0.1668* 


[0.0941] 


[0.0922] 

Hispanic 

-0.0089 

01 

-0.5124 


[0.0425] 


[0.3189] 

OtherRace 

-0.1616** 

02 

-0.0173 


[0.0747] 


[0.0733] 

Married 

0.1532*** 

04 

0.0063 


[0.0503] 


[0.0733] 

Divorced 

0.2381** 

05 

0.0376 


[0.1010] 


[0.0939] 

Age 

-0.0155 

06 

0.2965* 


[0.0106] 


[0.1679] 

DEPS 

-0.0208 

met_apc 1 

-0.1146 


[0.0169] 


[0.0775] 

YrsFrUGrad 

0.0071 

met_apc2 

0.0831 


[0.0128] 


[0.1070] 

MarineCorps 

0.4011*** 

met_apc3 

-0.2127** 


[0.1088] 


[0.0836] 

Army 

0.0519 

ServiceAcademy 

0.1491*** 


[0.1499] 


[0.0519] 

AirForce 

0.0824 

PrivateNFP 

0.0481 


[0.0838] 


[0.0585] 

Surface 

-0.3383*** 

PrivateFP 

0.0561 


[0.0744] 


[0.0742] 

Submarine 

-0.4216*** 

GSEAS 

-0.1760** 


[0.1178] 


[0.0809] 

Aviation 

-0.2784* 

GSOIS 

-0.2681*** 


[0.1522] 


[0.0958] 

Intelligence 

0.1317 

SIGS 

-0.1286** 


[0.1710] 


[0.0645] 

GroundCombat 

-0.0967 

_Y Start Acad_2007 

0.1820 


[0.0702] 


[0.1243] 

Naval_Aviation 

0.3649** 

_Y S tart Acad_2008 

0.2367** 


[0.1497] 


[0.1182] 

Marine_Aviation 

0.0369 

_YStartAcad_2009 

0.2836** 


[0.1754] 


[0.1311] 

Arrny_Aviation 

0.0836 

_Y Start Acad_2010 

0.0439 


[0.1957] 


[0.1555] 

AirForce_Aviation 

0.2127 

_YStartAcad_2011 

0.2971** 


[0.1592] 


[0.1405] 

Navy_Intelligence 

-0.1799 

_Y Start Acad_2012 

0.2636** 


[0.1855] 


[0.1309] 

Marine_Intelligence 

-0.5109** 

_YStartAcad_2013 

0.2170* 


[0.2178] 


[0.1257] 

Armv Intelligence 

-0.1358 

OStartOuar 2 

0.0213 


[0.1791] 


[0.0662] 

Navy_GroundCombat 

-0.1763 

OStartOuar 3 

-0.0386 


[0.1396] 


[0.0748] 

Marine_GroundCombat 

-0.4583*** 

OStartOuar 4 

0.0616 


[0.1556] 


[0.0853] 

Army_GroundCombat 

0.0465 

Constant 

4 3355*** 


[0.1349] 


[0.2906] 

Marine_Support 

-0.6372*** 




[0.2242] 

Observations 

5,289 



Adjusted R-squared 

0.190 


Robust standard errors in brackets 
*** p<0.01, ** p<0.05, * p<0.1 
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APPENDIX C. LPM REGRESSION RESULTS FOR THE OUTCOME 

GRADUATE 


GRADUATE 


VARIABLE 

COEFFICIENT 

VARIABLE COEFFICIENT 




dl 

-0.2232*** 

Marine Support 

-0.2033** 


[0.0272] 


[0.1034] 

Female 

' -0.0402 

Armv Support 

0.0775 


[0.0395] 

[0.1547] 

Black 

-0.1105** 

[0.0508] 

AirForce_Support 

-0.2313*** 

[0.0553] 

Hispanic 

-0.0877 

01 

0.0037 


[0.0570] 


[0.0949] 

OtherRace 

-0.0490 

02 

0.0617 


[0.0327] 

[0.0421] 

Married 

0.0461 

04 

-0.0071 


[0.0310] 


[0.0276] 

Divorced 

' 0.0540 

05 

-0.1396*** 


[0.0494] 

[0.0534] 

Age 

" -0.0039 

06 

0.1984 


[0.0035] 

[0.1250] 

DEPS 

-0.0116 

met apcl 

-0.0613* 


[0.0086] 


[0.0336] 

YrsFrUGrad 

" 0.0005 

met_apc2 

0.0688** 


[0.0038] 


[0.0345] 

Marin eCoips 

0.0630 

met_apc3 

-0.0861** 


[0.0884] 

[0.0373] 

Army 

" -0.1442 

ServiceAcademy 

0.0331 


[0.1253] 

[0.0204] 

AirForce 

0.0850* 

PrivateNFP 

-0.0052 


[0.0465] 

[0.0261] 

Surface 

-0.1457*** 

PrivateFP 

-0.0906 


[0.0296] 


[0.1429] 

Submarine 

-0.2015*** 

GSEAS 

-0.1128*** 


[0.0370] 


[0.0296] 

Aviation 

0.0487 

GSOIS 

-0.0981*** 


[0.0803] 

[0.0298] 

Intelligence 

-0.0344 

SIGS 

-0.0446 


[0.0722] 


[0.0277] 

GroundCombat 

-0.0621 

_YStartAcad_2007 

-0.0039 


[0.0473] 


[0.0369] 

Naval_Aviation 

-0.0465 

_YStartAcad_2008 

0.0199 


[0.0826] 


[0.0353] 

Marine_Aviation 

-0.0563 

YStartAcad 2009 

0.0301 


[0.1189] 


[0.0414] 

Army_Aviation 

-0.0966 

YStartAcad 2010 

-0.0384 


[0.1303] 

[0.0432] 

AirForce_Aviation 

-0.1370* 

_YStartAcad_2011 

-0.0026 

Navv Intelliaence 

[0.0804] 

-0.0081 

YStartAcad 2012 

[0.0414] 

-0.0903** 


[0.0793] 


[0.0403] 

Marine_Intelligence 

-0.0172 

YStartAcad 2013 

-0.6134*** 


[0.1304] 

[0.0383] 

Armv Intelliaence 

0.0831 

OStartOuar 2 

-0.0254 


[0.0908] 


[0.0268] 

Navy_GroimdCombat 

-0.0194 

_QStartQuar_3 

-0.0831*** 


[0.0612] 

[0.0311] 

Marine_GroundCombat 

-0.1658 

_QStartQuar_4 

-0.1225*** 


[0.1376] 


[0.0298] 

Army_GroundCombat 

0.2331** 

Constant 

1.3504*** 


[0.1181] 

[0.1157] 






Observations 

5,289 



Adjusted R-squared 

0.364 


Robust standard errors in brackets 


***p<0.01, ** p<0.05, *p<0.1 
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APPENDIX D. LPM REGRESSION RESULTS FOR THE OUTCOME 

PROMOTED 


_PROMOTED_ 

VARIABLE COEEHCIENT VARIABLE COEFFICIENT 


dl 

-0.1253*** 

M arine_S upport 

-0.3884*** 


[0.0276] 


[0.0769] 

Female 

-0.0157 

Army_Support 

-0.0272 


[0.0484] 


[0.0788] 

Black 

-0.0408 

AirForce_Support 

-0.2065 


[0.0374] 


[0.1262] 

Hispanic 

-0.0138 

01 

0.2753*** 


[0.0379] 


[0.0958] 

OtherRace 

-0.0435 

02 

-0.1268*** 


[0.0292] 


[0.0390] 

Married 

0.0846*** 

04 

-0.0873*** 


[0.0307] 


[0.0287] 

Divorced 

0.0514 

05 

-0.4005*** 


[0.0698] 


[0.0460] 

Age 

-0.0101*** 

06 

-0.2171** 


[0.0036] 


[0.0964] 

DEPS 

-0.0104 

met apcl 

0.0058 


[0.0085] 


[0.0407] 

YrsFrUGrad 

0.0018 

met_apc2 

0.0233 


[0.0041] 


[0.0255] 

MarineCorps 

0.2038*** 

met_apc3 

-0.0111 


[0.0647] 


[0.0396] 

Army 

-0.1308* 

ServiceAcademy 

0.0022 


[0.0683] 


[0.0223] 

AirForce 

0.0739 

PrivateNFP 

0.0016 


[0.1270] 


[0.0263] 

Surface 

-0.2172*** 

PrivateFP 

0.0727 


[0.0302] 


[0.0660] 

Submarine 

-0.0764* 

GSEAS 

-0.0628** 


[0.0404] 


[0.0296] 

Aviation 

-0.9628*** 

GSOIS 

-0.0736*** 


[0.2008] 


[0.0245] 

Intelligence 

-0.1469 

SIGS 

0.0349 


[0.1739] 


[0.0236] 

GroundCombat 

0.1565 

_YStartAcad_2007 

-0.0345 


[0.1832] 


[0.0450] 

Naval_Aviation 

0.8828*** 

_Y StartAcad_2008 

-0.0930** 


[0.2001] 


[0.0416] 

Marine_Aviation 

0.6711*** 

_YStartAcad_2009 

-0.2989*** 


[0.2160] 


[0.0425] 

Army_Aviation 

0.9687*** 

_Y Start Acad_2010 

-0.3362*** 


[0.2136] 


[0.0431] 

AirForce_Aviation 

1.1520*** 

_YStartAcad_2011 

-0.4546*** 


[0.2398] 


[0.0365] 

Navy Intelligence 

-0.0653 

YStartAcad 2012 

-0.5902*** 


[0.1778] 


[0.0340] 

Marine_lntelligence 

-0.2166 

_YStartAcad_2013 

-0.6678*** 


[0.1906] 


[0.0318] 

Army_Intelligence 

0.1968 

OStartOuar 2 

-0.0675*** 


[0.1782] 


[0.0255] 

Navy_GroundCombat 

-0.1288 

OStartOuar 3 

-0.0176 


[0.1880] 


[0.0277] 

Marine_GroundCombat 

-0.5448*** 

_QStaitQuar_4 

-0.1001*** 


[0.1983] 


[0.0256] 

Army_GroundCombat 

-0.1518 

Constant 

1.2456*** 


[0.1930] 


[0.1198] 



Observations 

5,289 



Adjusted R-squared 

0.358 

Robust standard errors in brackets 


*** p<0.01, 

** p<0.05, * p<0.1 
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APPENDIX E. LPM REGRESSION RESULTS FOR THE OUTCOME 

SEPARATED 


SEPARATED 


VARIABLES 

COEFFICIENT 

VARIABLE 

COEFFICIENT 

dl 

0.0562*** 

Marine Support 

0.3069*** 


[0.0203] 


[0.0723] 

Female 

" -0.0214 

Army_Support 

-0.2274** 


[0.0323] 


[0.1072] 

Black 

" -0.0069 

AirForce Support 

0.0162 


[0.0258] 


[0.1387] 

Hispanic 

-0.0648** 

01 

-0.1167*** 


[0.0317] 


[0.0371] 

OtherRace 

0.0607* 

02 

0.1238*** 


[0.0359] 


[0.0433] 

Married 

-0.1100*** 

04 

0.0045 


[0.0264] 


[0.0260] 

Divorced 

' -0.0852 

05 

0.1098* 


[0.0808] 


[0.0599] 

Age 

0.0063* 

06 

0.0051 


[0.0035] 


[0.0663] 

DEPS 

" 0.0092 

met_apc 1 

" 0.0181 


[0.0076] 


[0.0428] 

YrsFrUGrad 

-0.0068* 

met_apc2 

' -0.0272 


[0.0040] 


[0.0237] 

MarineCorps 

-0.1182*** 

met_apc3 

-0.0699** 


[0.0421] 


[0.0347] 

Army 

0.2211** 

ServiceAcademy 

0.0365 


[0.1083] 


[0.0225] 

AirForce 

' 0.0397 

PrivateNFP 

" -0.0305 


[0.1381] 


[0.0251] 

Surface 

0.0182 

PrivateFP 

-0.1167*** 


[0.0222] 


[0.0387] 

Submarine 

" 0.0377 

GSEAS 

" -0.0386 


[0.0247] 


[0.0257] 

Aviation 

0.0943 

GSOIS 

-0.0098 


[0.1037] 


[0.0255] 

Intelligence 

0.3509 

SIGS 

0.0414 


[0.2547] 


[0.0263] 

GroundCombat 

-0.1320*** 

YStartAcad 2007 

-0.0091 


[0.0448] 


[0.0408] 

Naval_Aviation 

-0.0109 

YStartAcad 2008 

0.0165 


[0.1068] 


[0.0388] 

Marine_Aviation 

0.2246* 

YStartAcad 2009 

-0.0502 


[0.1253] 


[0.0369] 

Army_Aviation 

-0 3499*** 

_YStartAcad_2010 

-0.0241 


[0.1347] 


[0.0385] 

AirForce_Aviation 

-0.1695 

YStartAcad 2011 

-0.0524 


[0.1809] 


[0.0358] 

Navv Intelligence 

-0.2964 

YStartAcad 2012 

-0.0196 


[0.2559] 


[0.0350] 

Marine_Intelligence 

-0.1722 

_YStartAcad_2013 

-0.1518*** 


[0.2543] 


[0.0278] 

Army_Intelligence 

-0.6392** 

_QStartQuar_2 

-0.0265 


[0.2642] 


[0.0202] 

Navy_GroundCombat 

0.1064** 

_QStartQuar_3 

-0.0672** 


[0.0518] 


[0.0297] 

Marine_GroundCombat 

0.2748*** 

_QStartQuar_4 

-0.0517*** 


[0.0813] 


[0.0190] 

Army_GroundCombat 

0.2897*** 

Constant 

0.0775 


[0.1049] 


[0.1175] 







Observations 

5,289 



Adjusted R-squared 

0.368 


Robust standard errors in brackets 


*** p<0.01, ** p<0.05, * p<0.1 
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APPENDIX F. OLS REGRESSION RESULTS FOR THE OUTCOME 

TQPR—HETEROGENEITY 
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APPENDIX G. LPM REGRESSION RESULTS FOR THE OUTCOME 
GRADUATE—HETEROGENEITY 
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APPENDIX H. LPM REGRESSION RESULTS FOR THE OUTCOME 
PROMOTED—HETEROGENEITY 
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APPENDIX I. LPM REGRESSION RESULTS EOR THE OUTCOME 
SEPARATED—HETEROGENEITY 
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